Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Make subrecipients into JSON explicitly #355

Merged
merged 3 commits into from
Jul 9, 2024
Merged

Make subrecipients into JSON explicitly #355

merged 3 commits into from
Jul 9, 2024

Conversation

laurelkorwin
Copy link
Contributor

@laurelkorwin laurelkorwin commented Jul 8, 2024 •
edited
Loading

  • Mapping each subrecipient to a JSON serializable object with .model_dumps so we can json.dumps() it in validate_workbook.py

@laurelkorwin laurelkorwin requested a review from as1729 July 8, 2024 21:40
@laurelkorwin laurelkorwin linked an issue Jul 8, 2024 that may be closed by this pull request
Treasury Report - Save subrecipient on upload validation #345
Closed
3 tasks
@laurelkorwin laurelkorwin marked this pull request as draft July 8, 2024 21:41
@github-actions github-actions bot added the python label Jul 8, 2024
@github-actions GitHub Actions
Copy link

github-actions bot commented Jul 8, 2024 •
edited
Loading

Terraform Summary

Step Result
🖌 Terraform Format & Style ✅
⚙️ Terraform Initialization ✅
🤖 Terraform Validation ✅
📖 Terraform Plan ✅

Hint: If "Terraform Format & Style" failed, run terraform fmt -recursive from the terraform/ directory and commit the results.

Output

Validation Output 
Success! The configuration is valid.
Plan Summary
CHANGE RESOURCE
add aws_s3_object.origin_dist_artifact["static/js/app.d6967b1b.js"]
aws_s3_object.origin_dist_artifact["static/js/app.d6967b1b.js.LICENSE.txt"]
update aws_ecs_service.console
aws_s3_object.lambda_artifact-graphql
aws_s3_object.lambda_artifact-processValidationJson
aws_s3_object.origin_dist_artifact["200.html"]
aws_s3_object.origin_dist_artifact["build-manifest.json"]
aws_s3_object.origin_dist_artifact["chunk-references.json"]
aws_s3_object.origin_dist_artifact["index.html"]
module.lambda_function-cpfValidation.aws_lambda_function.this[0]
module.lambda_function-graphql.aws_lambda_function.this[0]
module.lambda_function-processValidationJson.aws_lambda_function.this[0]
recreate aws_ecs_task_definition.console
aws_s3_object.lambda_artifact-python
module.lambda_function-cpfValidation.aws_lambda_permission.current_version_triggers["S3BucketNotification"]
module.lambda_function-graphql.aws_lambda_permission.current_version_triggers["APIGateway"]
module.lambda_function-processValidationJson.aws_lambda_permission.current_version_triggers["S3BucketNotification"]
delete aws_s3_object.origin_dist_artifact["static/js/app.c3e411c4.js"]
aws_s3_object.origin_dist_artifact["static/js/app.c3e411c4.js.LICENSE.txt"]

Pusher: @laurelkorwin, Action: pull_request_target, Workflow: Continuous Integration

@github-actions GitHub Actions
Copy link

github-actions bot commented Jul 8, 2024 •
edited
Loading

QA Summary

See our documentation for tips on how to resolve failing QA checks.

QA Check Result
🌐 Web Tests ✅
🔗 API Tests ✅
🐍 Python Tests ✅
📏 ESLint ✅
🧼 Ruff ✅
🧹 TFLint ✅

Test Coverage

Coverage report for api suite
St File % Stmts % Branch % Funcs % Lines Uncovered Line #s
🔴 All files 43.37 25.25 47.98 43.81
🔴  src 0 100 0 0
🔴   server.ts 0 100 0 0 6-13
🟢  src/directives/requireAuth 100 100 100 100
🟢   requireAuth.ts 100 100 100 100
🟡  src/directives/skipAuth 50 100 0 50
🟡   skipAuth.ts 50 100 0 50 13
🔴  src/functions 0 100 0 0
🔴   graphql.ts 0 100 0 0 15-27
🟡  src/functions/processValidationJson 56.86 50 60 56
🟢   processValidationJson.scenarios.ts 100 100 100 100
🟡   processValidationJson.ts 56 50 60 55.1 48-87,107-108,144-152,164-165,180-183,195-196,209
🔴  src/graphql 0 100 100 0
🔴   agencies.sdl.ts 0 100 100 0 1
🔴   expenditureCategories.sdl.ts 0 100 100 0 1
🔴   inputTemplates.sdl.ts 0 100 100 0 1
🔴   organizations.sdl.ts 0 100 100 0 1
🔴   outputTemplates.sdl.ts 0 100 100 0 1
🔴   projects.sdl.ts 0 100 100 0 1
🔴   reportingPeriods.sdl.ts 0 100 100 0 1
🔴   subrecipientUploads.sdl.ts 0 100 100 0 1
🔴   subrecipients.sdl.ts 0 100 100 0 1
🔴   uploadValidations.sdl.ts 0 100 100 0 1
🔴   uploads.sdl.ts 0 100 100 0 1
🔴   users.sdl.ts 0 100 100 0 1
🔴   validationRuleses.sdl.ts 0 100 100 0 1
🔴  src/lib 10.37 8.27 7.77 10.63
🟡   auth.ts 64.7 48.38 57.14 67.34 70-71,77-78,94-95,117,124,127,132-139,163,167
🔴   aws.ts 2.17 0 0 2.17 29-223
🟢   constants.ts 100 100 100 100
🔴   db.ts 45.45 50 50 45.45 15-35,41,43,50
🔴   ec-codes.ts 0 100 100 0 1
🟢   logger.ts 100 100 100 100
🔴   persist-upload.js 0 0 0 0 16-295
🔴   preconditions.ts 0 0 0 0 2-3
🔴   records.js 0 0 0 0 12-214
🔴   templateRules.ts 0 0 0 0
🔴   tracer.ts 0 100 100 0 5-14
🔴   validate-upload.js 0 0 0 0 18-790
🟢   validation-error.ts 83.33 100 50 83.33 22
🔴   validation-rules.js 0 0 0 0 6-194
🟡  src/services/agencies 67.34 50 80 67.34
🟢   agencies.scenarios.ts 100 100 100 100
🟡   agencies.ts 65.21 50 75 65.21 40-51,60-64,97-98,104,113-121
🟡  src/services/expenditureCategories 78.57 66.66 88.88 78.57
🟢   expenditureCategories.scenarios.ts 100 100 100 100
🟡   expenditureCategories.ts 77.77 66.66 88.88 77.77 30-34,49-52,60,91
🟡  src/services/inputTemplates 77.77 66.66 85.71 77.77
🟢   inputTemplates.scenarios.ts 100 100 100 100
🟡   inputTemplates.ts 76.92 66.66 85.71 76.92 25-29,39-40,50,85
🟡  src/services/organizations 67.39 87.5 50 67.39
🟢   organizations.scenarios.ts 100 100 100 100
🟡   organizations.ts 65.11 87.5 42.85 65.11 34-38,73,86-116,139-163
🟡  src/services/outputTemplates 77.77 66.66 85.71 77.77
🟢   outputTemplates.scenarios.ts 100 100 100 100
🟡   outputTemplates.ts 76.92 66.66 85.71 76.92 25-29,39-40,50,81
🟡  src/services/passage 74.07 62.5 100 74.07
🟡   passage.ts 74.07 62.5 100 74.07 18-19,65-76
🟡  src/services/projects 80 100 62.5 80
🟢   projects.scenarios.ts 100 100 100 100
🟡   projects.ts 78.57 100 62.5 78.57 45-51
🟡  src/services/reportingPeriods 67.5 70 50 67.5
🟢   reportingPeriods.scenarios.ts 100 100 100 100
🟡   reportingPeriods.ts 66.66 70 50 66.66 25-29,39-40,55-58,74,105-126
🟢  src/services/subrecipientUploads 92.3 100 83.33 92.3
🟢   subrecipientUploads.scenarios.ts 100 100 100 100
🟢   subrecipientUploads.ts 91.66 100 83.33 91.66 46
🟢  src/services/subrecipients 85.71 100 71.42 85.71
🟢   subrecipients.scenarios.ts 100 100 100 100
🟢   subrecipients.ts 84.61 100 71.42 84.61 47-52
🟡  src/services/uploadValidations 57.14 100 14.28 57.14
🟢   uploadValidations.scenarios.ts 100 100 100 100
🟡   uploadValidations.ts 53.84 100 14.28 53.84 10,16,30,38,45-48
🟢  src/services/uploads 86 66.66 78.26 86
🟢   uploads.scenarios.ts 100 100 100 100
🟢   uploads.ts 81.57 66.66 58.33 81.57 30,96,124-138
🟢  src/services/users 85.96 79.41 92 85.96
🟢   users.scenarios.ts 100 100 100 100
🟢   users.ts 84.9 79.41 88.88 84.9 227-229,238-242,260-261,275-278,296-298,306-307,312,321-324
🟢  src/services/validationRuleses 85.71 100 71.42 85.71
🟢   validationRuleses.scenarios.ts 100 100 100 100
🟢   validationRuleses.ts 84.61 100 71.42 84.61 43-48
Coverage report for web suite
St File % Stmts % Branch % Funcs % Lines Uncovered Line #s
🔴 All files 13.75 14.28 11.52 13.1
🟢  api/src/lib 100 100 100 100
🟢   constants.ts 100 100 100 100
🔴  web/src 28.57 18.75 66.66 28.57
🔴   App.tsx 0 0 0 0 3-36
🟢   Routes.tsx 100 100 100 100
🟡   auth.ts 50 50 100 50 19-24
🔴   entry.client.tsx 0 0 100 0 10-22
🔴  web/src/auth 7.14 0 4.16 7.14
🔴   localAuth.ts 9.09 0 8.33 9.09 39-68,76-80
🔴   passageAuth.ts 5 0 0 5 22-25,31-60
🔴  web/src/components/Agency/Agencies 0 100 0 0
🔴   Agencies.tsx 0 100 0 0 9-21
🔴  web/src/components/Agency/AgenciesCell 0 100 0 0
🔴   AgenciesCell.tsx 0 100 0 0 8-39
🔴  web/src/components/Agency/Agency 0 0 0 0
🔴   Agency.tsx 0 0 0 0 10-78
🔴  web/src/components/Agency/AgencyCell 0 100 0 0
🔴   AgencyCell.tsx 0 100 0 0 7-27
🔴  web/src/components/Agency/AgencyForm 0 0 0 0
🔴   AgencyForm.tsx 0 0 0 0 25-45
🔴  web/src/components/Agency/EditAgencyCell 0 100 0 0
🔴   EditAgencyCell.tsx 0 100 0 0 10-59
🔴  web/src/components/Agency/NewAgency 0 100 0 0
🔴   NewAgency.tsx 0 100 0 0 9-35
🟢  web/src/components/Navigation 100 37.5 100 100
🟢   Navigation.tsx 100 37.5 100 100 26-70
🔴  web/src/components/Organization/EditOrganizationCell 0 100 0 0
🔴   EditOrganizationCell.tsx 0 100 0 0 13-62
🔴  web/src/components/Organization/EditOrganizationForm 0 0 0 0
🔴   EditOrganizationForm.tsx 0 0 0 0 27-41
🔴  web/src/components/Organization/NewOrganization 0 100 0 0
🔴   NewOrganization.tsx 0 100 0 0 9-37
🔴  web/src/components/Organization/NewOrganizationForm 0 0 0 0
🔴   NewOrganizationForm.tsx 0 0 0 0 25-54
🔴  web/src/components/Organization/Organization 0 0 0 0
🔴   Organization.tsx 0 0 0 0 10-70
🔴  web/src/components/Organization/OrganizationCell 0 100 0 0
🔴   OrganizationCell.tsx 0 100 0 0 7-27
🔴  web/src/components/Organization/OrganizationPickListsCell 40 0 27.27 36.36
🟡   OrganizationPickListsCell.mock.ts 50 100 0 100
🔴   OrganizationPickListsCell.stories.tsx 0 0 0 0 6-32
🟡   OrganizationPickListsCell.tsx 64.28 100 50 58.33 14-16,51-76
🔴  web/src/components/Organization/Organizations 0 100 0 0
🔴   Organizations.tsx 0 100 0 0 9-21
🔴  web/src/components/Organization/OrganizationsCell 0 100 0 0
🔴   OrganizationsCell.tsx 0 100 0 0 8-37
🔴  web/src/components/ReportingPeriod/EditReportingPeriodCell 0 100 0 0
🔴   EditReportingPeriodCell.tsx 0 100 0 0 13-76
🔴  web/src/components/ReportingPeriod/NewReportingPeriod 0 100 0 0
🔴   NewReportingPeriod.tsx 0 100 0 0 9-35
🔴  web/src/components/ReportingPeriod/ReportingPeriod 0 0 0 0
🔴   ReportingPeriod.tsx 0 0 0 0 12-105
🔴  web/src/components/ReportingPeriod/ReportingPeriodCell 0 100 0 0
🔴   ReportingPeriodCell.tsx 0 100 0 0 7-34
🔴  web/src/components/ReportingPeriod/ReportingPeriodForm 0 0 0 0
🔴   ReportingPeriodForm.tsx 0 0 0 0 18-43
🔴  web/src/components/ReportingPeriod/ReportingPeriods 0 0 0 0
🔴   ReportingPeriods.tsx 0 0 0 0 13-98
🔴  web/src/components/ReportingPeriod/ReportingPeriodsCell 0 100 0 0
🔴   ReportingPeriodsCell.tsx 0 100 0 0 8-44
🟡  web/src/components/ReportingPeriodsCell 57.14 0 60 50
🟢   ReportingPeriodsCell.mock.ts 100 100 100 100
🔴   ReportingPeriodsCell.stories.tsx 0 0 0 0 6-32
🟢   ReportingPeriodsCell.tsx 100 100 100 100
🔴  web/src/components/TableBuilder 0 0 0 0
🔴   DebouncedInput.tsx 0 0 0 0 13-32
🔴   Filter.tsx 0 0 0 0 6-15
🔴   TableBuilder.tsx 0 0 0 0 22-70
🔴   TableHeader.tsx 0 0 0 0 5-42
🔴   TableRow.tsx 0 100 0 0 3-7
🟡  web/src/components/TemplateUploadReportingPeriodCell 55 0 55.55 47.05
🟢   TemplateUploadReportingPeriodCell.mock.ts 100 100 100 100
🔴   TemplateUploadReportingPeriodCell.stories.tsx 0 0 0 0 11-37
🟢   TemplateUploadReportingPeriodCell.tsx 100 100 100 100
🔴  web/src/components/Upload/EditUploadCell 0 100 0 0
🔴   EditUploadCell.tsx 0 100 0 0 10-66
🔴  web/src/components/Upload/NewUpload 0 100 0 0
🔴   NewUpload.tsx 0 100 0 0 7-35
🔴  web/src/components/Upload/Upload 0 0 0 0
🔴   Upload.stories.tsx 0 100 100 0 5-93
🔴   Upload.tsx 0 0 0 0 11-61
🔴  web/src/components/Upload/UploadCell 0 100 0 0
🔴   UploadCell.tsx 0 100 0 0 7-59
🔴  web/src/components/Upload/UploadForm 0 0 0 0
🔴   UploadForm.tsx 0 0 0 0 23-109
🔴  web/src/components/Upload/UploadValidationButtonGroup 0 0 0 0
🔴   UploadValidationButtonGroup.stories.tsx 0 100 0 0 5-47
🔴   UploadValidationButtonGroup.tsx 0 0 0 0 24-57
🔴  web/src/components/Upload/UploadValidationResultsTable 42.85 100 100 42.85
🔴   UploadValidationResultsTable.stories.tsx 0 100 100 0 6-41
🟢   UploadValidationResultsTable.tsx 100 100 100 100
🔴  web/src/components/Upload/UploadValidationStatus 0 0 0 0
🔴   UploadValidationStatus.tsx 0 0 0 0 3-30
🔴  web/src/components/Upload/Uploads 0 0 0 0
🔴   Uploads.tsx 0 100 0 0 9-17
🔴   columns.tsx 0 0 0 0 7-67
🔴  web/src/components/Upload/UploadsCell 0 100 0 0
🔴   UploadsCell.tsx 0 100 0 0 8-55
🔴  web/src/components/User/EditUserCell 0 100 0 0
🔴   EditUserCell.tsx 0 100 0 0 10-58
🔴  web/src/components/User/NewUser 0 100 0 0
🔴   NewUser.tsx 0 100 0 0 9-32
🔴  web/src/components/User/User 0 0 0 0
🔴   User.tsx 0 0 0 0 10-94
🔴  web/src/components/User/UserCell 0 100 0 0
🔴   UserCell.tsx 0 100 0 0 7-30
🔴  web/src/components/User/UserForm 0 0 0 0
🔴   UserForm.tsx 0 0 0 0 28-202
🔴  web/src/components/User/Users 0 100 0 0
🔴   Users.tsx 0 100 0 0 9-24
🔴  web/src/components/User/UsersCell 0 100 0 0
🔴   UsersCell.tsx 0 100 0 0 8-44
🟡  web/src/layouts/AuthenticatedLayout 60 50 100 60
🔴   AuthenticatedLayout.stories.tsx 0 100 100 0 5-13
🟢   AuthenticatedLayout.tsx 100 50 100 100 24
🟢  web/src/lib 100 100 100 100
🟢   formatters.tsx 100 100 100 100
🟢   seeds.ts 100 100 100 100
🔴  web/src/pages/Agency/AgenciesPage 0 100 0 0
🔴   AgenciesPage.tsx 0 100 0 0 8-12
🔴  web/src/pages/Agency/AgencyPage 0 100 0 0
🔴   AgencyPage.tsx 0 100 0 0 7-8
🔴  web/src/pages/Agency/EditAgencyPage 0 100 0 0
🔴   EditAgencyPage.tsx 0 100 0 0 7-8
🔴  web/src/pages...[Comment body truncated]

laurelkorwin
laurelkorwin commented Jul 9, 2024
View reviewed changes
python/src/schemas/schema_V2024_04_01.py
@@ -16,7 +16,7 @@
from src.schemas.project_types import NAME_BY_PROJECT


class StateAbbreviation(Enum):
class StateAbbreviation(str, Enum):
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Subclassing str for an Enum is a quick fix to make it JSON serializable

laurelkorwin
laurelkorwin commented Jul 9, 2024
View reviewed changes
python/src/lib/workbook_validator.py
@@ -283,6 +284,9 @@ def validate_workbook(workbook: Workbook) -> Tuple[Errors, Optional[str]]:
projects, subrecipients, version_string
)


subrecipients = [subrecipient.model_dump() for subrecipient in subrecipients]
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is the "supported" way in the latest version of Pydantic, see docs

I tested by hardcoding the same results object we use in validate_workbook in workbook_validator and running unit tests, then inspecting the output. json.dumps works and subrecipient output looks good
Screenshot 2024-07-08 at 8 26 06 PM

@laurelkorwin laurelkorwin marked this pull request as ready for review July 9, 2024 03:31
@as1729 as1729 merged commit 76bdb28 into main Jul 9, 2024
22 checks passed
@as1729 as1729 deleted the 345-treasury-report-save-subrecipient-on-upload-validation-3 branch July 9, 2024 14:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@as1729 as1729 as1729 approved these changes

Assignees
No one assigned
Labels
python
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Treasury Report - Save subrecipient on upload validation
2 participants
@laurelkorwin @as1729